4 ms·

While I feel like this is generally true for most programmers and knowledge workers, Git is absolutely not suited to the workflow of several industries, includi

While I feel like this is generally true for most programmers and knowledge workers, Git is absolutely not suited to the workflow of several industries, including the one I work in: games. Working with an engine like Unreal Engine for a project of any reasonable size requires working with both hundreds of thousands of active files (my current project's repo's HEAD has ~400k) and hundreds of gigabytes of active files, many of which are many GB on their own. Our repository (in perforce) is currently in the order of 10TB. Git, even with LFS and partial clone and shallow copies and fsnotify just falls apart at this scale. Add to that the necessity for less-technical people (artists, designers, VFX, audio, etc) to use it, and it is really just a non starter. I absolutely loathe Perforce (having used and occasionally admin'd it professionally since 2007), but I begrudgingly admit that is is currently the only publicly available VCS that can fulfill all of the requirements of this industry in a practical way. Plastic and the others just aren't there yet.

karmakaze 3y ago
This could be solved with references rather than copies if your tools integrated them. e.g. dependencies can be ref'd with source/name and version. To ensure availability all such used blobs could be stored efficiently elsewhere and versioned. It's great that Perforce works and I've heard others in the graphics field using it, so it satisfies a need. Don't know if/when it would be a general need for say GitHub users.
firesteelrain 3y ago
That’s why you need a binary repository like Artifactory and not store your large files in Git. But you can still track them in Git with the large files in a binary repository like Artifactory.
withinboredom 3y ago
This is a solved problem with git. I’ve worked on bigger projects than yours with histories that are so big you can’t clone them if you wanted to. I’ll make a note to drop what to google for tomorrow, but basically the history is fully available but git knows how to query it over the network. When you open a file, it loads it over the network. Remote build systems do your builds. Most of this was built by Microsoft to work on Windows with git. We moved from SVN in the late 2010’s, and holy crap man, did it change our workflows. No longer were we passing around patch files in a team for code reviews, but actually reviewing code somewhat like on GitHub. It was magical.
- IlPeach 3y ago
  I shed a tear as I remembered the pain of doing that with SVN
- est31 3y ago
  Microsoft had a bunch of solutions to handle their massive Windows repo: VFS for Git (GVFS), Scalar, and now it has a bunch of MS specific patches on top of the official git client, but apparently that one is also not required any more as partial clone is now supported on azure as well (which is another such implementation from Microsoft employees that made it to both GitHub and upstream git). So yeah, solved problem thanks to Microsoft. Solved multiple times in fact, and because it's Microsoft, only not all had Linux installers: https://github.com/microsoft/scalar/issues/323#issuecomment-927865734 https://github.com/microsoft/scalar/issues/323#issuecomment-... https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout https://github.blog/2020-01-17-bring-your-monorepo-down-to-s... https://devblogs.microsoft.com/devops/introducing-scalar/ https://devblogs.microsoft.com/devops/introducing-scalar/ https://github.com/microsoft/git https://github.com/microsoft/git https://devblogs.microsoft.com/devops/git-partial-clone-now-supported-in-azure-devops/ https://devblogs.microsoft.com/devops/git-partial-clone-now-...
  - withinboredom 3y ago
    Thanks!
- slalomskiing 3y ago
  > Add to that the necessity for less-technical people (artists, designers, VFX, audio, etc) to use it How is that solved with Git?
  - allarm 3y ago
    Maybe they just need to follow carefully written instructions created by more technical people?
    - 3y ago
Brian_K_White 3y ago
This sounds like a case of using excel as a db. Holding a hammer by it's head a declaring it unfit for normal people to paint with.
- afjeafaj848 3y ago
  I also work in games and everyone I know uses Perforce for the reasons mentioned above Have you considered that maybe they just have a different use case than you?
  - Brian_K_White 3y ago
    What I mean is, no one ever said that git was meant to hold bulk assets and resources directly, and so doing so is painting with a hammer, not an insufficient pain brush. Yes of course they have a different use case. That is just another way to say the same thing.

karmakaze 3y ago

This could be solved with references rather than copies if your tools integrated them. e.g. dependencies can be ref'd with source/name and version. To ensure availability all such used blobs could be stored efficiently elsewhere and versioned. It's great that Perforce works and I've heard others in the graphics field using it, so it satisfies a need. Don't know if/when it would be a general need for say GitHub users.

firesteelrain 3y ago

That’s why you need a binary repository like Artifactory and not store your large files in Git. But you can still track them in Git with the large files in a binary repository like Artifactory.

withinboredom 3y ago

This is a solved problem with git. I’ve worked on bigger projects than yours with histories that are so big you can’t clone them if you wanted to. I’ll make a note to drop what to google for tomorrow, but basically the history is fully available but git knows how to query it over the network. When you open a file, it loads it over the network. Remote build systems do your builds. Most of this was built by Microsoft to work on Windows with git. We moved from SVN in the late 2010’s, and holy crap man, did it change our workflows. No longer were we passing around patch files in a team for code reviews, but actually reviewing code somewhat like on GitHub. It was magical.

IlPeach 3y ago
I shed a tear as I remembered the pain of doing that with SVN
est31 3y ago
Microsoft had a bunch of solutions to handle their massive Windows repo: VFS for Git (GVFS), Scalar, and now it has a bunch of MS specific patches on top of the official git client, but apparently that one is also not required any more as partial clone is now supported on azure as well (which is another such implementation from Microsoft employees that made it to both GitHub and upstream git). So yeah, solved problem thanks to Microsoft. Solved multiple times in fact, and because it's Microsoft, only not all had Linux installers: https://github.com/microsoft/scalar/issues/323#issuecomment-927865734 https://github.com/microsoft/scalar/issues/323#issuecomment-... https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout https://github.blog/2020-01-17-bring-your-monorepo-down-to-s... https://devblogs.microsoft.com/devops/introducing-scalar/ https://devblogs.microsoft.com/devops/introducing-scalar/ https://github.com/microsoft/git https://github.com/microsoft/git https://devblogs.microsoft.com/devops/git-partial-clone-now-supported-in-azure-devops/ https://devblogs.microsoft.com/devops/git-partial-clone-now-...
- withinboredom 3y ago
  Thanks!
slalomskiing 3y ago
> Add to that the necessity for less-technical people (artists, designers, VFX, audio, etc) to use it How is that solved with Git?
- allarm 3y ago
  Maybe they just need to follow carefully written instructions created by more technical people?
  - 3y ago

Brian_K_White 3y ago

This sounds like a case of using excel as a db. Holding a hammer by it's head a declaring it unfit for normal people to paint with.

afjeafaj848 3y ago
I also work in games and everyone I know uses Perforce for the reasons mentioned above Have you considered that maybe they just have a different use case than you?
- Brian_K_White 3y ago
  What I mean is, no one ever said that git was meant to hold bulk assets and resources directly, and so doing so is painting with a hammer, not an insufficient pain brush. Yes of course they have a different use case. That is just another way to say the same thing.