4 ms·

This is exactly what I was thinking when I read the article. There is no reason for any Git repo to be that big. It's not a bug in Git, but more like a reasonab

This is exactly what I was thinking when I read the article. There is no reason for any Git repo to be that big. It's not a bug in Git, but more like a reasonable limit... if your project exceeds it, you're doing it wrong. I'm also curious as to how they used to do it without git, maybe using TFS? I wonder what the timings on that were. Anyway, I don't think GVFS is the way to go, and I hope that it either doesn't get accepted or doesn't play a role outside of Windows. It's good to see more Git usage, but hacking away instead of fixing the problematic project seems somewhat idiotic. I can imagine other tools having problems with a single project that size, are they going to hack those as well?

ctdonath 10y ago
"640k is enough for anyone."
BugsJustFindMe 10y ago
Ah, yes. The mating call of the wild elite computer expert. Microsoft: "We, only one of most technologically advanced companies with only the 2nd or 3rd highest market cap of any public company on the planet depending on the day, had a problem with infrastructure trying to manage possibly the largest software project that anyone has ever made. And then we solved it." You: "Stop doing what you're doing and pay attention to meeeeeeeeeeee."
- testUser69 10y ago
  I think it's more naive to believe that large companies always make the right decisions. We see Microsoft make mistake after mistake (look at windows 8, windows vista). The only time they fix their mistakes is when they're made public. So there is no reason to fix their mistake of a code base.
  - CJefferson 10y ago
    But, it seems all large companies (Facebook, Google, Microsoft) are using a mono-repo, and no-one is publicly admitting to using many small repos. If no company is using many small repos for truly massive projects, then it's hard to argue it would be a good idea. Could everyone who has looked at this problem make the wrong choice?
    - keithlfrost 10y ago
      Note that Amazon is not on your list. They did not adopt the mono-repo approach. Their tools have their own advantages and disadvantages, to be sure, but mono-repo is not the only way.
      - CJefferson 10y ago
        I hadn't heard about Amazon. That's good to know.
        krallja 10y ago
        Read Steve Yegge's Google vs Amazon rant: it's amazing. https://plus.google.com/+RipRowan/posts/eVeouesvaVX https://plus.google.com/+RipRowan/posts/eVeouesvaVX
    - Sacho 10y ago
      This is also mirrored by many not-so-big javascript projects, i.e. see the adoption of https://github.com/lerna/lerna https://github.com/lerna/lerna
    - aanm1988 10y ago
      > no-one is publicly admitting to using many small repos. Amazon. They built an entire system around managing versions so that they can make it work.
  - ska 10y ago
    It would be naive to assume large companies always make the right decisions, which is why few people do that. However, when you see that a large company is doing something in a way that you thing is silly or strange, the logical thing to ask first is "what do they understand that I don't understand?". It won't always be the case, of course, but most of the time it will turn out you were missing something. Assuming off the bat that they are idiots and you know a better way is staggeringly naive.
    - oz 10y ago
      >the logical thing to ask first is "what do they understand that I don't understand?". There's even a name for this: "Going from D to C - from disparagement to curiosity". I think I first heard it from @patio11.
      - ska 10y ago
        It isn't the snappiest label, but it does capture the idea well.
      - patio11 10y ago
        Doesn't sound like the way I would phrase the thought, so I checked: http://www.kalzumeus.com/2012/09/17/ramit-sethi-and-patrick-mckenzie-on-getting-your-first-consulting-client/ http://www.kalzumeus.com/2012/09/17/ramit-sethi-and-patrick-... <-- Ramit said it here, although I suspect it might be older.
deleted 10y ago
[deleted]
klodolph 10y ago
The problem with this argument is that it doesn't really explain why large repos are wrong. It turns out the choice between monorepo and multi repo does not have a one-size-fits-all answer. Microsoft can make their own tooling, and internal tools don't have to run outside of Microsoft so they're easier to develop. Google reportedly used to use Perforce for their monolithic codebase, and Facebook is supposed to use Mercurial with a bunch of modifications. They all have huge code bases mostly in one repo (I've heard Facebook had a >50GB Git repository, and Google's codebase is supposed to be in the TB range).
CJefferson 10y ago
I don't see why there should be limits. I recently ran some experiments, one file per experiment, one result file per experiment. At around 100,000 files, git started getting very upset. Why shouldn't I be allowed to have 100,000 files, or a million files, in a directory? Why should it be my job, as a user, to manually rearrange my data into a format my computer is happier with?
garysieling 10y ago
Being able to use git for things it's not designed for would be a strength, e.g. it works great as a database for certain kinds of projects. I put all the records for https://www.findlectures.com https://www.findlectures.com in it, because then I can use diff tools for testing changes. Obviously this is nowhere near the size of the Windows codebase, but I could see a world where GVFS would be helpful for collaboration on this project.

ctdonath 10y ago

"640k is enough for anyone."

BugsJustFindMe 10y ago

Ah, yes. The mating call of the wild elite computer expert. Microsoft: "We, only one of most technologically advanced companies with only the 2nd or 3rd highest market cap of any public company on the planet depending on the day, had a problem with infrastructure trying to manage possibly the largest software project that anyone has ever made. And then we solved it." You: "Stop doing what you're doing and pay attention to meeeeeeeeeeee."

testUser69 10y ago
I think it's more naive to believe that large companies always make the right decisions. We see Microsoft make mistake after mistake (look at windows 8, windows vista). The only time they fix their mistakes is when they're made public. So there is no reason to fix their mistake of a code base.
- CJefferson 10y ago
  But, it seems all large companies (Facebook, Google, Microsoft) are using a mono-repo, and no-one is publicly admitting to using many small repos. If no company is using many small repos for truly massive projects, then it's hard to argue it would be a good idea. Could everyone who has looked at this problem make the wrong choice?
  - keithlfrost 10y ago
    Note that Amazon is not on your list. They did not adopt the mono-repo approach. Their tools have their own advantages and disadvantages, to be sure, but mono-repo is not the only way.
    - CJefferson 10y ago
      I hadn't heard about Amazon. That's good to know.
      - krallja 10y ago
        Read Steve Yegge's Google vs Amazon rant: it's amazing. https://plus.google.com/+RipRowan/posts/eVeouesvaVX https://plus.google.com/+RipRowan/posts/eVeouesvaVX
  - Sacho 10y ago
    This is also mirrored by many not-so-big javascript projects, i.e. see the adoption of https://github.com/lerna/lerna https://github.com/lerna/lerna
  - aanm1988 10y ago
    > no-one is publicly admitting to using many small repos. Amazon. They built an entire system around managing versions so that they can make it work.
- ska 10y ago
  It would be naive to assume large companies always make the right decisions, which is why few people do that. However, when you see that a large company is doing something in a way that you thing is silly or strange, the logical thing to ask first is "what do they understand that I don't understand?". It won't always be the case, of course, but most of the time it will turn out you were missing something. Assuming off the bat that they are idiots and you know a better way is staggeringly naive.
  - oz 10y ago
    >the logical thing to ask first is "what do they understand that I don't understand?". There's even a name for this: "Going from D to C - from disparagement to curiosity". I think I first heard it from @patio11.
    - ska 10y ago
      It isn't the snappiest label, but it does capture the idea well.
    - patio11 10y ago
      Doesn't sound like the way I would phrase the thought, so I checked: http://www.kalzumeus.com/2012/09/17/ramit-sethi-and-patrick-mckenzie-on-getting-your-first-consulting-client/ http://www.kalzumeus.com/2012/09/17/ramit-sethi-and-patrick-... <-- Ramit said it here, although I suspect it might be older.

deleted 10y ago

[deleted]

klodolph 10y ago

The problem with this argument is that it doesn't really explain why large repos are wrong. It turns out the choice between monorepo and multi repo does not have a one-size-fits-all answer. Microsoft can make their own tooling, and internal tools don't have to run outside of Microsoft so they're easier to develop. Google reportedly used to use Perforce for their monolithic codebase, and Facebook is supposed to use Mercurial with a bunch of modifications. They all have huge code bases mostly in one repo (I've heard Facebook had a >50GB Git repository, and Google's codebase is supposed to be in the TB range).

CJefferson 10y ago

I don't see why there should be limits. I recently ran some experiments, one file per experiment, one result file per experiment. At around 100,000 files, git started getting very upset. Why shouldn't I be allowed to have 100,000 files, or a million files, in a directory? Why should it be my job, as a user, to manually rearrange my data into a format my computer is happier with?

garysieling 10y ago

Being able to use git for things it's not designed for would be a strength, e.g. it works great as a database for certain kinds of projects. I put all the records for https://www.findlectures.com https://www.findlectures.com in it, because then I can use diff tools for testing changes. Obviously this is nowhere near the size of the Windows codebase, but I could see a world where GVFS would be helpful for collaboration on this project.