11 ms·

I disagree that Git's biggest flaw is its lack of scalability. Cases where git needs to scale tend to be isolated to companies that have the manpower to build a

I disagree that Git's biggest flaw is its lack of scalability. Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google). Git's flaws are primarily in usability/UX. But I think for its purpose, functionality is far more important than a perfect UX. I'm perfectly happy knowing I might have to Google how to do something in Git as long as I can feel confident that Git will have the power to do whatever it is I'm trying to do. A competitor would need to do what git does as well as git does it, with a UX that is not just marginally better but categorically better, to unseat git. (Marginally better isn't strong enough to overcome incumbent use cases) And for the record: I think git-lfs issues are primarily usability issues, and tech improvements. The tech enhancements will be solved if there's enough desire, and as I mentioned the usability problems are more annoyances than actual problems.

rtpg 7y ago
Scaling isn't even just about number of files or size of them. A problem I've hit is just in having cross-repo stuff work well. Monorepos are helpful partly because git submodules are not ideal for good workflows, and splitting stuff across multiple git repos can backfire (it doesn't help that almost all the tooling around CI and the like is repo-based instead of project based). I would love a layer over Git to handle workflow issues related to multi-repo projects
- nsomaru 7y ago
  Sub modules have terrible UX and are completely counterintuitive. Subtrees work well for my small repos when I need to vendor something in
  - rossmohax 7y ago
    There is also a git subrepo from subtree author, worth checking out IMHO
MaulingMonkey 7y ago
> Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google). I constantly run into git scalability issues as an individual. I don't use any of the UI clients because they all fail hard on mostly-code git repositories. I abandoned my VisualRust port in part because the mere 100MB of mingw binaries involved for that meant it was using github LFS, which meant CI was hitting github quota limits, and as I wasn't part of the organization - nevermind an admin with billing rights - I couldn't even pay to up said quota limits paying out of pocket myself even if I wanted to. I'm not going to inflict git's command line experience - which confounds and confuses even seasoned programmers - on any of the less technical artists that might be employed at a typical gamedev shop, even if git might be able to scale acceptably if locally self-hosted at a single-digit employee shop. A few dozen or hundred employees? Forget it. Use perforce, even though it costs $$$, is far from perfect, and also has plenty of scaling issues eventually.
- pja 7y ago
  The fact that you had a problem with github quotas isn’t really a problem with git though, is it?
  - MaulingMonkey 7y ago
    The whole reason git lfs exists is to workaround git scalability problems. Its raison d'etre is problems with git. That one of - if not the - most popular tool to solve said git scalability problems, also has scalability problems in practice, is both ironic - and absolutely a problem with the git ecosystem. To be pithy - "Even the workarounds don't work." "Technically", you might say, "that specific symptom with git lfs, and that service provider, isn't the fault of git the command line tool, nor the git protocol". And you would be technically correct - which is the best kind of correct. But I don't think we're referring to either of those particularly specific things with "Git" when we ask the article's question of "Is Git Irreplacable?". I'm already the weirdo for using git the command line tool - most of my peers use alternative git UI clients, and I don't mean gitk. The git protocol is routinely eschewed in favor of zips or tarballs over HTTPS, Dropbox, Sneakernet, you name it - and is invisible enough to not be worth complaining about to pretty much every developer who isn't actively working on the backend of a git client or server. Not to mention it's been extended/replaced with incremental improvements over the years already. So I'm using a slightly broader definition of "git", inclusive of the wider ecosystem, that allows me to credit it for the alternative UI clients that do exist, rather than laughing off the question at face value - as something that has already been replaced.
Ace17 7y ago
I work for a 40-people game studio. A major limitation of git is how it deals with many "big" (~10Mb) binary files (3D models, textures, sounds, etc.). We ended up developing our own layer over git, and we're very happy ; even git-lfs can't provide similar benefits. This technique seems to be commonplace for game studios (e.g Naughty Dog, Bungee), so certainly git has room for improvement here.
- onion2k 7y ago
  Has any studio open sourced what they've built? Or turned it in to a product? It seems there could be an opportunity to do something before git solves the problems for that use case.
- rustybolt 7y ago
  Jup, I experienced this too (not at a game studio though, and the team I worked with wasn't nearly experienced enough to write a layer over git). When we switched to a new version of the git gui it would stop working because when you click through the GUI, it would perform some git operations that were supposed to run fast. I filed an issue that quickly got shot down with 'wontfix, your repo is too large and git is not for binary files'.
- taftster 7y ago
  Do you have tools that you can utilize diffs from your binary file changes? Or does a change simply just replace all the bytes. I'd argue if it's the later, that git was never the right choice to begin with. You don't really want to record a full 10MB of data every time you change one pixel in your texture or one blip in your sound, right? So I don't know if this is a "major limitation" of git per se. Not saying there's a better solution off-the-shelf (you're obviously happy with your home grown). But this was probably never a realistic use for git in the first place.
  - MaulingMonkey 7y ago
    While I can't speak for the person you're replying to, the technology at least exists. Binary diffs are sometimes used to distribute game updates, where you're saving on bandwidth for thousands if not millions of players - which costs enough $$$ to actually be worth optimizing for. On the other hand, between simpler designs and content encryption being sometimes at odds with content compression... so is just sending the full 10MB. For a VCS - I'd probably be happy enough to just have storage compression - using any of the standard tools on the combination. > You don't really want to record a full 10MB of data every time you change one pixel in your texture or one blip in your sound, right? Actual changes to content in a gamedev studio are very unlikely to be as small as a single pixel. Changes to source code are unlikely to be as small as a single character either. And we definitely want a record of that 10MB. We're willing to sacrifice some of our CI build history. Maybe only keeping ~weekly archives, or milestone/QAed builds after awhile, of dozens or hundreds of GB - and maybe eventually getting rid of some of the really old ones eventually. Having an exact binary copy of a build a bug was reported against can be incredibly useful.
chme 7y ago
> I disagree that Git's biggest flaw is its lack of scalability. Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google). I would say that the sole thing git was developed for, the Linux Kernel, is (starting to be) painful to work with when using git.
- taftster 7y ago
  The Linux Kernel is big, but it's not likely as big (in terms of lines of code or pick your metric) as Google or Microsoft repositories. Maybe the kernel is just starting to feel that pain? Honestly asking.. Do you speak from some level of authority that the Linux kernel is stretching the boundaries of git? Or are you just saying that more speculatively? What is the painful part?
jeltz 7y ago
Maybe I am a weirdo but I have always thought that git's UI is very intuitive (with some exceptions like sub modules). SVN on the other hand was an unintutive mess where I had to look up commands all the time.
- jMyles 7y ago
  Agreed - I had been looking around this thread like, "<slow blink> - surely I'm not the only one that finds git to be a rewarding exercise in teamwork?"
jariel 7y ago
Fully agree. The magic sweet spot might be the fact that most projects to not need to be distributed. This is where a lot of complexity is derived. So without all those extra concerns - and - a more elegant UI framework (i.e. rational commands) - and possibly something that scales a little better. That's enough mojo to unseat git for a lot of things.
signal11 7y ago
> Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google). I recollect that for Windows (which also uses git), MS have actually extended git with "Git Virtual File System" rather than replace it[1]. But I do agree that broadly, not everyone needs the scale. [1] https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/ https://devblogs.microsoft.com/bharry/the-largest-git-repo-o...

rtpg 7y ago

Scaling isn't even just about number of files or size of them. A problem I've hit is just in having cross-repo stuff work well. Monorepos are helpful partly because git submodules are not ideal for good workflows, and splitting stuff across multiple git repos can backfire (it doesn't help that almost all the tooling around CI and the like is repo-based instead of project based). I would love a layer over Git to handle workflow issues related to multi-repo projects

nsomaru 7y ago
Sub modules have terrible UX and are completely counterintuitive. Subtrees work well for my small repos when I need to vendor something in
- rossmohax 7y ago
  There is also a git subrepo from subtree author, worth checking out IMHO

MaulingMonkey 7y ago

> Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google). I constantly run into git scalability issues as an individual. I don't use any of the UI clients because they all fail hard on mostly-code git repositories. I abandoned my VisualRust port in part because the mere 100MB of mingw binaries involved for that meant it was using github LFS, which meant CI was hitting github quota limits, and as I wasn't part of the organization - nevermind an admin with billing rights - I couldn't even pay to up said quota limits paying out of pocket myself even if I wanted to. I'm not going to inflict git's command line experience - which confounds and confuses even seasoned programmers - on any of the less technical artists that might be employed at a typical gamedev shop, even if git might be able to scale acceptably if locally self-hosted at a single-digit employee shop. A few dozen or hundred employees? Forget it. Use perforce, even though it costs $$$, is far from perfect, and also has plenty of scaling issues eventually.

pja 7y ago
The fact that you had a problem with github quotas isn’t really a problem with git though, is it?
- MaulingMonkey 7y ago
  The whole reason git lfs exists is to workaround git scalability problems. Its raison d'etre is problems with git. That one of - if not the - most popular tool to solve said git scalability problems, also has scalability problems in practice, is both ironic - and absolutely a problem with the git ecosystem. To be pithy - "Even the workarounds don't work." "Technically", you might say, "that specific symptom with git lfs, and that service provider, isn't the fault of git the command line tool, nor the git protocol". And you would be technically correct - which is the best kind of correct. But I don't think we're referring to either of those particularly specific things with "Git" when we ask the article's question of "Is Git Irreplacable?". I'm already the weirdo for using git the command line tool - most of my peers use alternative git UI clients, and I don't mean gitk. The git protocol is routinely eschewed in favor of zips or tarballs over HTTPS, Dropbox, Sneakernet, you name it - and is invisible enough to not be worth complaining about to pretty much every developer who isn't actively working on the backend of a git client or server. Not to mention it's been extended/replaced with incremental improvements over the years already. So I'm using a slightly broader definition of "git", inclusive of the wider ecosystem, that allows me to credit it for the alternative UI clients that do exist, rather than laughing off the question at face value - as something that has already been replaced.

Ace17 7y ago

I work for a 40-people game studio. A major limitation of git is how it deals with many "big" (~10Mb) binary files (3D models, textures, sounds, etc.). We ended up developing our own layer over git, and we're very happy ; even git-lfs can't provide similar benefits. This technique seems to be commonplace for game studios (e.g Naughty Dog, Bungee), so certainly git has room for improvement here.

onion2k 7y ago
Has any studio open sourced what they've built? Or turned it in to a product? It seems there could be an opportunity to do something before git solves the problems for that use case.
rustybolt 7y ago
Jup, I experienced this too (not at a game studio though, and the team I worked with wasn't nearly experienced enough to write a layer over git). When we switched to a new version of the git gui it would stop working because when you click through the GUI, it would perform some git operations that were supposed to run fast. I filed an issue that quickly got shot down with 'wontfix, your repo is too large and git is not for binary files'.
taftster 7y ago
Do you have tools that you can utilize diffs from your binary file changes? Or does a change simply just replace all the bytes. I'd argue if it's the later, that git was never the right choice to begin with. You don't really want to record a full 10MB of data every time you change one pixel in your texture or one blip in your sound, right? So I don't know if this is a "major limitation" of git per se. Not saying there's a better solution off-the-shelf (you're obviously happy with your home grown). But this was probably never a realistic use for git in the first place.
- MaulingMonkey 7y ago
  While I can't speak for the person you're replying to, the technology at least exists. Binary diffs are sometimes used to distribute game updates, where you're saving on bandwidth for thousands if not millions of players - which costs enough $$$ to actually be worth optimizing for. On the other hand, between simpler designs and content encryption being sometimes at odds with content compression... so is just sending the full 10MB. For a VCS - I'd probably be happy enough to just have storage compression - using any of the standard tools on the combination. > You don't really want to record a full 10MB of data every time you change one pixel in your texture or one blip in your sound, right? Actual changes to content in a gamedev studio are very unlikely to be as small as a single pixel. Changes to source code are unlikely to be as small as a single character either. And we definitely want a record of that 10MB. We're willing to sacrifice some of our CI build history. Maybe only keeping ~weekly archives, or milestone/QAed builds after awhile, of dozens or hundreds of GB - and maybe eventually getting rid of some of the really old ones eventually. Having an exact binary copy of a build a bug was reported against can be incredibly useful.

chme 7y ago

> I disagree that Git's biggest flaw is its lack of scalability. Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google). I would say that the sole thing git was developed for, the Linux Kernel, is (starting to be) painful to work with when using git.

taftster 7y ago
The Linux Kernel is big, but it's not likely as big (in terms of lines of code or pick your metric) as Google or Microsoft repositories. Maybe the kernel is just starting to feel that pain? Honestly asking.. Do you speak from some level of authority that the Linux kernel is stretching the boundaries of git? Or are you just saying that more speculatively? What is the painful part?

jeltz 7y ago

Maybe I am a weirdo but I have always thought that git's UI is very intuitive (with some exceptions like sub modules). SVN on the other hand was an unintutive mess where I had to look up commands all the time.

jMyles 7y ago
Agreed - I had been looking around this thread like, "<slow blink> - surely I'm not the only one that finds git to be a rewarding exercise in teamwork?"

jariel 7y ago

Fully agree. The magic sweet spot might be the fact that most projects to not need to be distributed. This is where a lot of complexity is derived. So without all those extra concerns - and - a more elegant UI framework (i.e. rational commands) - and possibly something that scales a little better. That's enough mojo to unseat git for a lot of things.

signal11 7y ago

> Cases where git needs to scale tend to be isolated to companies that have the manpower to build a finely-tuned replacement (see: MS, Google). I recollect that for Windows (which also uses git), MS have actually extended git with "Git Virtual File System" rather than replace it[1]. But I do agree that broadly, not everyone needs the scale. [1] https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/ https://devblogs.microsoft.com/bharry/the-largest-git-repo-o...